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Abstract 


Do students emotions and behaviours influence their learning in mathematics classes, and if yes, how and 
why? Based on a qualitative research design, evidence on how the use of Electronic Voting Systems (EVS) 
has influenced students’ emotional and behavioural engagement with university mathematics learning is 
presented. The results show that the use of EVS enables students to diagnose their level of understanding 
of learning material, relative to the rest of the class. EVS use also helps, in real time, to stimulate, 
maintain, and re-focus student attention during mathematics instruction. A favourable perception of the 
learning environment as being altruistic appears to have influenced students to emotionally identify and 
align themselves with mathematics learning goals. There is enhanced interactivity in EVS-facilitated 
lectures, including positive student competitiveness, while the EVS anonymity utility also enhances 
increased student participation rates. However, a major disincentive to student engagement is the use 
of EVS-based mathematics questions with low cognitive demand. This research is a contribution to how 
specific educational technology-based interventions influence student engagement. 

Key words: emotional/behavioural engagement, digital natives, EVS/Clickers, engineering mathematics. 


Introduction 


Technology is transforming the student learning experience. Across Europe, Africa, 
Asia and North America (e.g., Howard, 2012; Jaschik & Lederman, 2014; Keppell, Suddaby 
& Hard, 2015; Parr, 2013; Nkurunziza, 2013; Akue-Kpakpo, 2013; OECD, 2013, 2015a), and 
through innovative platforms, such as virtual schools and universities (Guo, Kim & Rubin, 
2014), open educational resources (Nikoi & Armellini, 2012), big data and learning analytics 
(Oblinger, 2012; Petersen, 2012), and interactive polling (Voelkel & Bennett, 2014; King & 
Robinson, 2009a,b), technology is transforming both the student experience or engagement 
and associated learning outcomes (Stover et al., 2015; Bryne-Davis et al., 2015; Brady, Seli 
& Rosenthal, 2013; OECD, 2015b) across all subjects and disciplines (Puschmann & Bastos, 
2015; Dalbello, 2011; Shapero, 2015; Thomson, Purcell & Rainee, 2013; Olive & Makar, 2009; 
Di Blas et al., 2014). 

Moreover, the current and future generations of students or digital natives, who do not 
know what it means to live without the internet have ‘a completely different type of relationship 
to the media’ (Wong, 2015). The students we teach and can expect to teach are emotive and 
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have an ‘engrained instinct to share everything, including their personal data’ (Elkin, 2015). For 
example, ‘blogs have led to a complete reversal of the “diary” phenomenon — whereas once kids 
kept their feelings locked up in a book, today they (or at least many of them) prefer to post them 
online for all to see and share’ (Prensky, 2012, p. 91). Other examples of digital natives’ social 
media activities include “liking”, “unliking”, selfies (personal photographs), and interactive 
commenting on friends’ feeds or activities. Hence, the engagement mode of the digital native 
student population with technology has a quite visible emotional component. 

In contrast, the default mode of technology integration in undergraduate education 
eschews overt displays of emotion, while prioritizing cognitive engagement with learning, thus 
inadvertently enabling ‘flight from feelings’ (Gibbs, 2013, p. 12; see also Suarez-Orozco et al., 
2015, and Zins & Elias, 2006) classrooms, especially at the university level. Yet recent research 
studies strongly indicate that the affective learning domain is indispensable to catalysing 
learning transfer or sustainable learning (Salcito, 2012; Stover et al., 2015; OECD, 2015b; 
Heckman & Kautz, 2014), and that considerations of student emotions and behaviours can 
create a “gateway to learning” effect on student learning outcomes (Cazzell & Rodriguez, 
2011). So the goal in this article is to demonstrate through a qualitative research-based case 
study, how an educational technology such as interactive polling can be used to catalyse student 
engagement with learning mathematics through a description of the impact of the use of the 
technology on students’ emotions and behaviour (1.e., the affective learning domain). 


Case Study on the Impact of Technology on Affective Learning 


This case study is therefore presented to highlight the potential and critical role that 
technology can play in catalysing digital natives’ emotional and behavioural engagement with 
learning in both future classrooms and informal, out-of-school-or-university contexts. Therefore, 
this article will be focusing on the most neglected aspect of university student learning — the 
affective domain (Cazzell & Rodriguez, 2011). Specifically, the article will highlight, through 
a case study, the impact of interactive polling (clicker) technology on students’ behaviour and 
emotions, as a measure of student engagement with learning mathematics. 


Introduction: Electronic Voting Systems 


EVS is an educational tool that can be used in a class, at its most basic level, in the 
same manner polling devices are used on the television programme, ‘Who Wants to Be a 
Millionaire’ (Ask the Audience section). The EVS technology typically consists of handsets, 
which are similar in shape, design and function to small television remote control units; a 
receiver or dongle that is essentially a USB device that is plugged into a computer port; and the 
enabling software which has to be downloaded to a computer. A lecturer uses the software on 
the computer to create (usually multiple choice) questions, which can be presented to students 
in class. The students answer the questions by clicking the corresponding alphanumeric answer 
choice on their EVS handsets (Figure 1). 

Student responses are then displayed, also in real time, in the form of a suitable chart 
(Figure 1). The lecturer may then decide to elaborate on any relevant issues arising out of the 
question and answer display session. 
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Figure 1: Chart showing the distribution of students’ answers to an EVS question. 


EVS is therefore a technology that affords an instructor the means to give students, 
especially ina large class, the opportunity to engage with course material by having them answer 
questions during a lecture, with subsequent provision of feedback to students. Moreover, this 
study specifically considers the use of EVS to support formative assessment, by aligning the 
EVS-based mathematics questions used with the anticipated student learning outcomes (e.g., 
see King& Robinson, 2009b; Beatty et al., 2006). 


Correlation between the Use of Electronic Voting Systems and Student 
Engagement 


In the most comprehensive review of the student engagement literature to date, 
Fredricks, Blumenfeld and Paris (2004) delineated engagement into the following three distinct 
yet interconnected dimensions of engagement: Cognitive, Behavioural and Emotional (p. 3; 
see also Duckworth & Yeager, 2015 and OECD, 2015b), with the Behavioural and Emotional 
Engagement constructs constituting the affective learning domain. 

Student engagement is particularly critical in mathematics because it is a learning field 
that students often struggle with, and one that frequently generates feelings of apathy, alienation 
and inadequacy in students. This is one reason why there has been a proliferation of mathematics 
support centres in universities across the UK (e.g., Croft, 2002) and the USA (e.g., Shaqlaih 
& Celik, 2013; Ellis & Berry, 2012) to provide extra resources for students struggling with the 
mathematical components of their university courses. Moreover, research strongly indicates that 
engagement (Table 1) has a highly positive correlation with increase in student achievement 
and attendance or retention rates (e.g., Helme & Clarke, 2001; Fredricks, Blumenfeld, & Paris, 
2004; Blumenfeld & Meece, 1988; Marks, 2000; Connell et al., 1995; Boyle et al., 2001; King 
& Robinson, 2009a). Research in mathematics education may therefore be ‘strengthened if 
researchers integrate affective issues into studies of cognition and instruction’ (McLeod, 1992, 
p. 575). 
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Further, research indicates that the use of EVS could significantly increase student 
engagement (e.g., Brady, Seli & Rosentahl, 2013; Byrne-Davis et al., 2015; Caldwell, 2007, 
Simpson & Oliver, 2007; Fies & Marshall, 2006). One study concluded that the most significant 
benefit of EVS use is the catalysis of student engagement with learning (King& Robinson, 
2009a). 

However, student engagement, specifically in relation to EVS use, has often been 
defined narrowly; with studies purportedly evaluating engagement mostly focusing on the 
impact of EVS use on student attitudes or participation in class (e.g., see Simpson & Oliver, 
2007). Consequently, this article will highlight the relationship between the use of EVS and the 
affective learning domain in stimulating student engagement with learning by focusing on the 
following more comprehensive emotional and behavioural engagement constructs: 

= Student Attention 

= Student Comparison of Academic Performance 

= Surface (Lecturer-Student), Peer and Technical Interactivity 

« Instruction to Stimulate Positive Learning Challenge 

= Student Perception and Response to Altruistic Instruction, and 

=» Anonymity 

= Disadvantages of Using EVS 

= Engagement Interconnectedness 


Significance of Research 


One of the conclusions of the comprehensive review of engagement literature was the 
need for ‘studies of interventions’ (Fredricks, Blumenfeld, & Paris, 2004, p. 87) that may help 
illuminate the ‘development or malleability of engagement’ (p. 87) to specific educational 
interventions. This study meets this need by highlighting the use of EVS and associated 
questions to facilitate emotional and behavioural engagement within the disciplinary domain of 
university mathematics. 

Further, the authors also called for the adoption of a qualitative approach to measuring 
engagement, which would provide ‘thick descriptions of classroom contexts’ about the factors 
that influence engagement (p. 86). Again, this study meets this criterion as the descriptions of 
engagement provided are based on evidence from interviews with students, supplemented by 
classroom observations and observations of students working on specific tasks in the interviews 
conducted. 


Table 1. Aspects of student learning that have been influenced by high levels of 
student engagement (Source: Fredricks, Blumenfeld, and Paris (2004), 
Boyle et al., (2001), and King and Robinson (2009a). 





#1 Academic achievement 

#2 Student boredom and disaffection 
#3 Attendance and retention rates 
#4 Attitude to learning 

#5 In-class participation 











Methodology of Research 


To answer the research question, ‘How has the use of EVS impacted (or otherwise) 
student engagement with respect to the learning of mathematics at university?’, a research 
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study predicated on a qualitative research methodology was conducted. Further, a semi- 
structured interview approach was employed to interview 10 volunteer students. As a result of 
the preliminary studies conducted (e.g., King & Robinson, 2009a,b), substantial knowledge on 
the ways that EVS use may influence student learning had been acquired. However, there were 
still gaps in that knowledge. Therefore, the semi-structured interview approach was adopted 
as the most effective means of addressing these ‘gaps’ (Cohen, Manion & Morrison, 2007, p. 
354). Further, this research assumes an interpretivist approach because this study is focused on 
presenting the interpretations of respondents about learning mathematics with EVS. Therefore, 
the research accounts presented are constructions rather than definitive conceptualisations of 
respondent learning experiences with EVS. 


Sample and Sampling 


The target population for the research was second-year Automotive and Aeronautical 
Engineering students (n = 150) who were taught an engineering mathematics module in the 
2008/2009 academic year; an explanation about why this research is still relevant seven years 
later is presented in the Discussion section. Six male and four female Automotive/Aeronautical 
Engineering students volunteered for the interviews (the synonyms, K1 to K10, have been 
adopted to protect the privacy of the 10 student interviewees). The Engineering students 
were identified as the primary target for purposive sampling principally because this cohort 
of second-year engineering students had been introduced to the use of EVS in their first year 
mathematics modules. It could thus be expected that their views on EVS use would be more 
mature or at least be immune to an extent from the novelty effect of EVS use, than students who 
had just been introduced to EVS. Therefore, the sampling approach that was implemented was 
purposive sampling (Bryman, 2008, p. 458). 


Method 


Semi-structured interviews were adopted to interview the 10 Engineering students who 
volunteered to participate in the research. This approach is sufficiently structured to enable 
consistency, 1.e., all interviewees were presented with the same sets of questions, and yet flexible 
enough to capture nuances and unanticipated students’ comments and insights (Bryman, 2008, 
p. 438). A sample of the interview questions is presented in Table 2. 
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36| Table 2. The interview questions. 








#1 GENERAL QUESTIONS 


« What are the disadvantages/problems (if any) of using EVS in lectures? 

« What are the benefits (if any) of using EVS in lectures? 

+ Were there times you did not vote in class? 

* Does EVS help you think more about a question or topic? How? 

* How do you feel when an EVS question (i.e., because it’s too hard or easy) puts you on the spot? 
« Any other comments? 


#2 QUESTIONS ON FEEDBACK 


« What happens when you get questions wrong? What do you (usually) do then or later? 

+ What are your views about the [instructor] feedback you get? Does it help you identify clearly where you went 
wrong, show you how you can get it right the next time, enough time for feedback? 

+ — Multiple select effect? 

* Response range effect? 


#3 SPECIFIC EVS MATHEMATICS QUESTIONS 


+ What's your approach to solving maths questions? To EVS-type questions? 
* Do you recall this question (e.g., Figure 3) 

* How would you solve this problem? 

+ What happens when you see a question on double integrals? 

+ Which aspect (if any) of the questions did you have difficulty with? 

















The interview questions also included specific EVS questions (e.g., see ‘#3’ in Table 2, 
and also Figure 1) that had been used during lectures for the engineering mathematics module 
class, on which all the interviewees were enrolled. The questions were based on the topics that 
had been covered in class, including Multiple Integration, Fourier Series and Vector Calculus. 
The main rationale for the inclusion of the questions was to assess whether and/or how the 
use of EVS questions had influenced student engagement with learning on the engineering 
mathematics course. 


Analysis of Interview Data 


To analyse the data from student interviews, thematic analysis was adopted, and 
this facilitated the identification and grouping of relevant themes and sub-themes, with the 
organisation, structure and presentation of the themes dictated, on a macro scale, by the 
research question, and at the micro level, by the interview questions. Using questions to guide 
the analysis ‘is a very useful way of organising data, as it draws together all the relevant data 
for the exact issue of concern to the researcher, and preserves the coherence of the material’ 
(Cohen, Manion & Morrison, 2007, p. 468). The analytical structure adopted is presented in 
Table 3. 
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Table 3. A simplified description of the analysis structure adapted for this study. 





Overview of the Data Analysis Protocol 





¢ #1 Research and interview question-guided thematic analysis approach (Cohen, Manion & Morrison, 2007:468; 
Bryman, 2008:554) 

¢ #2 Data collection intentionally structured to meet research objectives via adoption of semi-structured interview 
approach and design of interview protocol 

¢ #3 Exhaustive reading and manual annotation of individual interviewee transcripts 

¢ #4 Coding of interview data with respect to the three macro themes — Cognitive, Behavioural and Emotional 
Engagement 

¢ #5 Coding of interview data with respect to the constituent micro themes under each of the three macro themes 
(e.g. micro themes under Cognitive Engagement — Deliberate Practice, Relative Performance, etc; Behavioural 
Engagement — Attention; Emotional Engagement - Perception of Learning Environment) 





Trustworthiness 


To determine the reliability and objectivity of the research, the following Trustworthiness 
criteria (i.e., Lincoln & Guba, 1985): Dependability (which parallels Reliability) and 
Confirmability (which parallels Objectivity) were adopted (see also Bryman 2008, p. 377). 

To ensure dependability and confirmability, an audit approach, 1.e., “keeping complete 
records of all phases of the research process’ (Lincoln & Guba, 1985, p. 317), was adopted. 
These records include the spreadsheet/table used in recruiting interviewees, the interview 
protocol consisting of all the materials and questions used during the interviews, audiotapes of 
the interviews in virtual formats, transcripts of all interviews, documents detailing the thematic 
analysis techniques used in making theoretical inferences and the documents (i.e., analysis 
templates) produced as a result, and draft copies of this article. Further, draft versions of this 
article have been submitted to relevant academic personnel for review, with the manuscript 
updated based on the feedback received. 


Other Validity Measures 


To ensure face validity for the interviews, the interview materials were submitted to 
senior faculty members for review. Sections of the data analysis protocol had also been piloted 
with another group of students (Sports Technology) at the host university. 

To minimise bias due to the acquiescence factor (Cohen, Manion & Morrison, 2007, 
p. 151), interviewees were reassured at the beginning of each interview session that all their 
contributions would be treated with confidentiality. For example, all student interviewees had to 
sign a Consent Form that guaranteed their privacy, and freedom to withdraw from the research 
study at any time during the research process. Further, it is essential to clarify that the researcher 
who interviewed the students had not taught or planned to teach them or their class, and also 
did not have direct access to these students As indicated in Table 2, the interview protocol often 
directly required students to elaborate on the perceived disadvantages of using EVS, and not 
just focus on the benefits. 


Results of Research 
Behavioural Engagement - Student Attention 


The research results that are presented in this section will focus on how the use of 
electronic voting systems has influenced three dimensions of student attention. 
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Stimulating Student Attention 


In the excerpt below, K9 talked about how the sight of EVS handsets at the beginning 
of a lecture and the association of the handsets with questions to be asked in class created a 
feeling of ‘anticipation’ or eagerness about the lecture. In fact, s/he prefaced this by implying 
that the atmosphere or ambience around a ‘voting system lecture’ was remarkably different 
from conventional lectures in a positive way: 


K9: You can tell when, we’ve had, like, a voting system lecture, because everyone comes into 
the room and sees the voting system. And everyones sort of a bit more on edge, and, you know, 
anticipation about, ‘oh, I’ve got to pay attention because, like, there’ going to be a question 
coming up’. Whereas before, it was just, like, ‘oh, I’m just copying down this note, and the next 
note, and the next note’. 


K8 also talked about how EVS use made students ‘focus on the lecture’, and contrasted this 
with the scenario in other lectures where students were in the habit of falling asleep in class (and 
implied that this did not happen in EVS-enabled lectures): 


K8: Mm, when you compare a similar scenario with another lecture where they don t use voting 
systems, people just come to the lecture and sleep, you know? To be honest. And during voting 
systems, at least that doesn t happen, you know? Even if they’re not listening when the lecture ’s 
going on... 

I: Yes. [“T” is the interviewer, i.e., the author] 

KS8: When they see a question, they start working it out, you know? As in, they start, you know, on 
the paper, they start something, you know? So it’s better than not doing anything at all. 


Maintaining Student Attention 


The results indicate that EVS use, especially when the questions are spread throughout 
a lecture, may help maintain student attention. In the excerpt below, K1 implied, in a positive 
way, that the instructor often deliberately asked students questions just ‘to make sure you’ve 
been listening’, in order to ostensibly maintain student attention: 


K1: Yeah, just at any time during the lecture, [instructor] ’ll just spring us, bring a question up, 
so, to make sure you've been listening. 


Similarly, K4 talked about how the timing and the use of EVS questions helped ‘break the 
lecture up a bit’, and how this helped his/her concentration in class or to put it another way, got 
‘his/her mind active again’: 


K4: They break it, they break the lecture up a bit and I find, that obviously your concentration 
can then be distracted away from the visual, from learning on the board. And then you can 
actually do something, do something yourself, get your mind active again and then obviously you 
can listen for the next 20 minutes again. 


Meanwhile, K2 stated that s/he found that the usually limited time allocated for voting often 
helped him/her to concentrate on a question in order to get the right answer: 


K2: It can only help for you to concentrate. Because the thing is you know that you’re being 
given 30 seconds to solve this, for example. You’re being given 30 seconds and you have to 
concentrate and then you have to vote. So it is related. ‘Cause you... Because of the time limit 
you have to concentrate on the question and then you have to vote quickly so that you get it right. 
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Re-Focusing Student Attention 


In the excerpt below, K4 talked about how getting a question wrong was a sort of reality 


check that brought home the message, ‘I actually haven’t listened at all today’, and how this 
tended to make him/her feel more accountable about paying attention in class: 


I: ... but this one, you're given a question maybe after As just done a topic and A wants to find 
out how well you got it. 

K4: Yeah. It makes you realise, like, if you’ve actually been listening or if you ve just been sitting 
there, and, you know, on auto-pilot. And, like sometimes when A’ put a question up, I’m like 
‘T actually haven t listened at all today’. Like, what am I doing, why am I here if I’m not going 
to listen. Like, it kind of makes you, like, it kind of holds you accountable in a way, like, your 
attention span. 


Getting a question wrong in class seemed to have the same effect on K7 as it had on K4. 


K7 stated that getting a question wrong would make him/her think, ‘I should have paid more 
attention’. However, s/he pointed out that this was usually followed by the desire to make up 
for the inattention by reviewing his/her notes alongside the feedback from the instructor: 


I: What about, if before a question came up, for one reason or another, you were switched off 
and thinking about something else. You were not paying attention to that particular topic and a 
question comes up — what happens? 

K7: Um, [always think ‘T should have paid attention’, but then it helps because I'll have to look 
through the notes I’ve taken, and if I still don t quite get it, then A will go through it and then, it 
just makes it really. 


Summary: Student Attention 


In summary, the use of EVS for mathematics instruction in real time has helped to 


stimulate, maintain and re-focus student attention in class in the following three ways: 


EVS use helps to stimulate student attention. EVS use makes students pay attention 
during a lecture because they know they are going to be asked a question, usually about 
the topic presented in class. 

EVS use helps to maintain student attention. EVS questions may be used as a way of 
managing a lecture by presenting the questions at carefully selected intervals in order 
to defuse boredom and monotony. 

EVS use helps to re-focus student attention. If a student had not been paying attention 
prior to the administration of a question, this serves as a wake-up call to the student i.e. 
highlights the inattention and also what is required to remedy the situation. 


Behavioural Engagement - Student Comparison of Academic Performance 


Another feature of EVS usage, the display of answers to a question in a format that is 


viewed by the whole class (e.g., Figures 2 and 3), helps illustrate how an individual student may 
use this statistic to diagnose their level of understanding and performance relative to the class. 
In his/her submission, on the pre-interview questionnaire for instance, K1 talks about the power 
of the EVS answer display spread to illustrate relative performance: 


K1: Allows you to answer questions to see if you are at the same level as the rest of the group. 
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This information about relative performance provides a powerful diagnostic tool to the 
student — a reference point. It could be argued that a feedback type that includes this reference 
score is much more useful to students than those without (e.g. individual test scores or grades 
without any reference points, teacher claims about student performance without any referential 
benchmarks, etc.). For example, K2 stated that information about relative performance was 
critical because the ‘competitive’ nature of the class required this knowledge: 


I: You said, ‘It lets you know where you need to work more’, for instance — why is that important? 
K2: Its important that you... you know, if you’re doing engineering, its very competitive, so you 
always have to know how the other students are doing. And by that you actually know, and then 
sometimes you actually feel good about yourself that you did well. 


Similarly, K4 clarified the importance of having information about how the rest of a class 
performed on an EVS question, by commenting that access to this information would either 
bring reassurance or highlight the need to take corrective measures: 


K4: It just, like, ‘cause if you’re constantly getting a different answer from everyone else, you’re 
the one with the little bar going ‘4% answered this’ and it’s wrong, or then on the other hand, 
like, say, you're the right answer and only a few people have got it, it makes you kind of, like, 
feel good about where you are. 

I: Yes 

K4: But like, in general, like, most of the time you all get the same answer and it makes you 
feel. It reassures you that you're, like, on the same pace as the rest of them. I think if you were 
struggling, say, if you were constantly getting a different answer to everyone else. It would make 
you, like, It’s kind of anonymous as well, it would make you, it would make it easier for you to go 
and approach the teacher and say ‘look, you know, I’m struggling here’. 


Moreover, K4’s comments seem to suggest that information about areas of ‘struggle’ 
would trigger proactive action from the student with respect to approaching the instructor outside 
the classroom. This is perhaps an indication of how information about relative performance, 
which is predicated on deliberate practice, may be a catalyst (Draper, 2009) for learning beyond 
the classroom, especially to address gaps in understanding. 


Behavioural Engagement - Interactivity 


Interactivity in EVS-enabled lectures refers to the use of questions to actively engage 
students in the learning process, instead of them being passive recipients of information 
transmission from the lecturer. Based on classroom observations and interview data, EVS use 
has the potential to catalyse two levels or modes of interactivity in mathematics classrooms. 
These interactivity modes are: Lecturer-Student Interactivity, Peer Interactivity, and Technical 
Interactivity. 


Lecturer-Student (Surface) Interactivity 


This is the interaction between the student(s) and instructor, i.e., instructor poses a 
question which students respond to by answering, and which the instructor then responds to by 
providing feedback. This sets up a chain of interactivity between student and instructor, which 
is not a typical feature of contemporary university mathematics lectures. But this occurs at the 
surface level because the mode of communication is limited, and mostly instructor-driven and/ 
or initiated. 
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In the excerpt below, K8 talked about how the use of EVS ‘changed’ what usually 
happens in a mathematics lecture, in that students had the opportunity of interacting with the 
instructor, via the EVS question and answer sessions: 


K8: Because of the others’ interaction, you know? There 5 interaction between the students and 
the teacher. 

I: What kind of interaction? 

K8: Um, answering. Like, what basically happens if A asks a question... we'll be asked to 
answer. 

So, directly, we may not be answering, but we use the voting system. 

I: You've got that contact to give a response back. 

K8: Exactly. 

I: So, that doesn t happen in other lectures? 

K8: No, it doesn t. Very few. It’s only when a person raises an arm and asks a question is there 
interaction between the student and the teacher. 


K5 had also written on the pre-interview questionnaire, that one of the ways that EVS 
had ‘helped’ was that it provided an avenue for ‘more interaction with the lecturer’. More 
evidence for surface interactivity is not presented here because research evidence (e.g., Simpson 
& Oliver, 2007) clearly indicates that this is the most common form of interactivity enabled by 
the use of EVS. 


Peer Interactivity 


This is the interaction between students. Class observations show that this interaction 
occurs when students compare their answers with their neighbours after voting, talk through a 
problem with a classmate before voting, or in a limited sense, on the few occasions when the 
instructor asks the students to discuss their answers with their neighbours. 

The instructor would often encourage students to talk to their peers in order to challenge 
one another about the rationale for the answer choices selected in response to an EVS question. 
However, the instructor would typically only instigate this kind of discussion when the class 
response indicated a lack of consensus e.g. a question in which only 50% of the class got it 
right, as K9 describes in the excerpt below. It should be noted that in this particular class, 
students often sat with their friends, and so peer interactivity in this context is contingent on 
existing group dynamics. Students were more likely to chat with their friends knowing that 
they would not be ridiculed or embarrassed if their knowledge of the mathematical topic under 
discussion was below par: 


K9: Whereas if its more sort of 50/50, then occasionally I'll say ‘ok, who voted for this, who 
voted for this and then talk amongst yourselves... to sort of try and convince the other person 
why it’s that one’. And then we'll go over it so you know roughly why, even if the answer was 
wrong from your part, then you'll know why the other person thought that answer was right 
or... So it’s quite, it’s quite good talking about it every now and then, ‘cause even if the answer s 
wrong, you know why someone would have voted for that. 

eh ce I found it, mm, quite helpful because normally I sit with my friends, so even if my answer 
wrong, it didn t really matter. 

....Um, I dont think it would work so well if, you know, you're split into groups or anything, 
because I dont, I know other people in my class, in my group are cleverer than me, so it’s not 
always nice knowing that they’re always right and you’re always wrong. 

But when you, when it’s with your friends, it, it’s quite nice to think ‘oh well, they voted for that 
one for a reason’... 
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Generally, most interviewees appeared to talk to their peers about the EVS questions, 
only after they had worked on a problem and submitted their own answers. K5, for instance, 
describes how s/he would vote and afterwards ask a peer, ‘what did you get?’ S/he also alluded 
to the group dynamics inherent in the classes where ‘it’s usually small groups of us [i.e. friends] 
sitting together’ and ‘looking at each other’s work’ to discover if anyone had a different answer 
and probably talking about why that was so: 


K5: Um, usually I would vote first and then ask them ‘what did you get?’ 

abyeveuse) Yes. I think most of the time what tends to happen in our lectures is — I dont know 
whether it’s just us — but its usually small groups of us sitting together...... usually we would 
answer and then, um, if one of us, or if we get a different answer, we look at each other's work 
and work out how, you know, who has gone wrong and why. 


K10 also talked about how s/he would vote first and only then talk to his/her peers to 
avoid being influenced: 


I: Are there times in class when you wanted to vote, you talked to the guy sitting next to you or 
the guy sit, your mate or people who seem to be close to you before you answer the question or 
after you answer the question? 

K10: Um, I think, you kind of, I go and answer it first, and if you talk to somebody else, in case 
you can change it, you’ve always got the option of — 

I: So you’re just asking ‘what have you got?’ 

K10: You're, like, being influenced by other people. 


It should be noted that although some level of student-student interaction 1.e. students 
talking to each other in class occurs at least sporadically in most conventional mathematics 
classes, EVS use assures a higher frequency/structure and more learning-focused occurrences 
of this type of interactivity. In the excerpt below, K9 stated that although s/he usually sat with a 
group of friends, s/he would ‘normally’ vote first, and only talk about how s/he voted afterwards 
with his/her friends: 


K9: Normally, normally we'd click...... and then, like, talk to the people next to us and say ‘oh, I 
got, I said B’ Um, but normally we vote and then... ... talk about it afterwards. 


Peer interactivity appears to infuse an element of positive learning challenge or 
competitiveness into the voting process. In the excerpt below, K4 talked about how s/he would 
often wait impatiently for the instructor to close the polling and for the results to be revealed 
because of his/her anticipation or excitement at seeing whether s/he got the answer right. 
Similarly, s/he talked about how s/he would ask for the answers that his/her friends selected as 
a sort of competition to see whose answer would be revealed to be the right one: 


K4: I want to know if I got it right or not, you know. 

I: Yeah. Ok. So theres anticipation to know whether you got it right or wrong. 

K4: Yeah. It’s exciting, it’s like, ‘oh god’. 

I: T’d better get that right... how the rest of the class do?’ 

K4: Yeah. I’d be like ‘I got that right, yeah’. Or, like, before your friends are done, you're like 
‘what did you get or what did you get’ or you've got two different answers, it’s like ‘I bet I’m 
right, I bet I’m right’ and it’ like ‘yes’. 


K8 also talked about how this element of challenge when s/he stated that s/he would 


often ask his/her friend, after voting, about the answer s/he had selected ‘just to see if s/he was 
right or wrong’: 
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I: Ok. After you've voted, you dont ask them to see ‘what did you get, do you, what was your 
answer?’ You dont, you dont usually ask? 

K8: Um, before the answer comes up or after the answer comes up? 

I: Before or after. 

K8: After, I do. 

I: You do. 

K8: After. When it, suppose we get 75% for A and 35% for B... ... or something like that, you 
know? 

Then I'll ask the person sitting next to me what he got, you know? Just to make sure, just to see 
what he would have thought, you know? Just to see if he was right or wrong. 

I: Ok. Why do you want to know that? 

K&: Just ‘cause he’s my friend, so - 


Technical Interactivity 


This may be defined as the interaction between students and the EVS handset — the 
students get to ‘do something’, although this usually translates into merely clicking a handset. 
Its significance might be that the physical action of a clicking a handset could perhaps help 
reignite student interest or attention. As this form of interactivity, has a limited impact on 
behavioural engagement, no evidence of this interraction mode will be presented. 

In summary, another facet of behavioural engagement that EVS use has influenced is the 
level or mode of interactivity in lectures. The question-and-answer sessions provide a level of 
instructor-student interaction that is otherwise absent in typical mathematics lectures. Moreover, 
the question sessions are often structured in a way that allows students to interact with each 
other. One expression of this is the element of challenge that peer interaction introduces into 
classroom dynamics, i.e., students want to see if they can do better than their peers. 


Emotional Engagement 


How has EVS use influenced how students fee/ about engaging with university 
mathematics instruction? In this section, evidence on how EVS use has impacted emotional 
engagement, with reference to how students perceived the instructional environment, will be 
presented. 


Student Perception and Response to Altruistic Instruction 


Research on EVS indicates that students typically have positive attitudes about the use 
of EVS in mathematics lectures (e.g., Simpson & Oliver, 2007; Bruff, 2009), so this section is 
not about that aspect of emotional engagement. Instead, I will focus on how students appear to 
perceive an EVS-enabled instructional environment as one that is to a large extent intentionally 
or altruistically designed to aid their learning. 

For example, K7 implicitly alluded to this favourable perception of the altruistic nature of 
the instructional environment when s/he stated that s/he would only adopt educated guesswork 
in answering questions in class, so as not to provide the instructor with unreliable feedback on 
student progress: 


I: You dont like selecting the ‘I dont know’ option [i.e., to discourage guesswork]. 

K7: No I dont. I'd rather, even ifit’s an educated guess, I don t like guessing unless it's educated, 
because, especially as [instructor] does use it quite often to see how we are doing and to see 
what we need to put more work into in class... 

I: Yes. 

K7: ... um, I think if you luckily guess the right answer and you don t know it, it could affect that if 
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many people have done it. And [instructor] might think ‘oh, the class knows what they ’re doing’ 
and we've guessed it. So I try not to guess if I can help it. 


Similarly, K4 reiterated the position that it is important for the instructor to have accurate 
feedback about student knowledge, based on an implicit assumption that the instruction is 
directed at helping them learn and review areas they might be struggling with: 


I: Suppose they have an ‘I dont know’ option — do you go for that instead of guessing? 

K4: No. I dont think ‘I dont know’ is very helpful on it. Like, not if you know anything. But I 
guess it is, ‘cause, like, it shows [instructor] who, if, like, people actually dont know. And it 
would probably make [instructor] more likely to go through a question if lots of people dont 
know. 


Further, the EVS systems are solely used for formative assessment. For example, K2 
stated that s/he usually took the EVS tests i.e. answering the questions in class seriously because 
‘that tells me where I’m standing in class’, although s/he acknowledged that not all students 
emulated his/her approach: 


I: And coursework too, if you do it well, it helps you with your final grade. But the [EVS-based 
mathematics] questions that get put up in class, you dont want to get graded for that. 

K2: No, you dont. 

I: So does that affect your attitude towards the questions? I mean, do you take them seriously or? 
K2: Yeah, I do take them seriously. Because that tells me where I’m standing in my class. 


Anonymity 


In the two excerpts below, K1 talks about how the anonymity aspect of EVS use helps 
with confidence and allays student fears about being put on the spot: 


I: And this question says ‘in what ways has the use of voting systems in MAB104 hindered or 
helped your learning of additional calculus?’ And you said it helps you to answer anonymously, 
so nobody needs to know what you... ... how you answered. Why is that important? 

K1: Um, because you dont always have the confidence to put your hand up if you dont know 
whether you're right or not. But it lets you do it without anybody else knowing. 

I: Or even, doesnt that put you on the spot sometimes? It’s been so long, and then it’s something 
you're also going to do in the future, so you might not have enough information. 

K1: It does. But because theres the anonymous handsets, it’s ok. I wouldnt like A to do this and 
then come over to you and say ‘right, whats the answer?’ Because that would put me on the 
spot. Because it’s anonymous and also because A puts ‘I don t know’ there as well as an option... 


The deployment of new or interactive technologies in new situations, whether for 
learning or other purposes, could be expected to evoke positive feelings due to the novelty 
of the technology. So it is pertinent to point out that the students whose comments about the 
emotional aspect of learning with EVS are being presented in this section are not new to the 
technology, having been taught in a first year university mathematics module by an instructor 
who used EVS in the class. Consequently, the novelty factor exerts much less influence on the 
feelings of students about EVS use presented here. 

In the excerpt below, K7 talks about how EVS use was introduced in a first year 
university mathematics module, and how s/he initially found the technology ‘intimidating’, but 
later became familiar with the technology because EVS was regularly used in class: 
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I: What did you like? Dont mention the names of the lecturers. What did you like about the two 
modules? What stood out for you, I mean, the way they lectured? 

K7: Um, they, um, the first one covered what I'd either done at A level, A level maths or Further 
Maths. 

I: So it was an addition? 

K7: Yes. It was a nice break from the other subjects which were brand new, and it nice to recap. 
Um, the second one was more new topics, which was nice, but also, um, we started using the 
voting systems from then. 

I: Ok. The one on voting systems, the one on voting systems, so this is not your first introduction 
to voting systems... 

K7: No. 


S/he also stated that s/he had talked about EVS use with peers from other universities 
who thought that classroom voting was a ‘good idea’: 


K7: At first it was a bit intimidating ‘cause you didn t know, the very first question, you didnt 
know how it was going to work... ... even though you're pressing it anonymously and obviously, 
it was anonymous. It was just things going through my head. But once we started using them 
and I found them helpful, it was, the beginning of every class, pick one up and just use them 
throughout. 

I: So what do you think - you used voting systems in first year, you're now using them in second 
year — how do you feel about this? Is it something you've welcomed, in the sense that part of your 
learning, I mean, you welcome it in a way now. 

K7: Yeah. I like it and I’ve mentioned it to people studying similar courses at other universities, 
and when I’ve mentioned that we vote in class, they’ve all been surprised ‘cause they dont do it. 
And they've said it sounds like a good idea and it’s definitely something I personally like. 


Disadvantages of Using EVS 


As part of the interview protocol, interviewees were asked to comment on any negative 
aspects or disadvantages of using EVS in lectures that they had experienced or observed. In 
addition, the interviewees also often commented during the course of the interviews, without 
prompting from the researcher, about aspects of EVS that they considered to be potentially 
disengaging or disadvantageous. Due to volume constraints, asummary of these disadvantages, 
based on descriptions provided by interviewees and supplemented by interviewee observations, 
is hereby presented: 

1. EVS Malfunction: These include technical, i.e, EVS handset and software 
malfunction. 

2. Non-Voting: This includes issues surrounding the logistics of distributing handsets 
before lectures, low student participation in some EVS QA sessions, and sufficiency 
of voting time; 

3. Usage: This includes issues around the perception of some questions/distracters 
as being sloppy or ineffectual, guesswork, student swapping of answers, flippant 
attitude towards voting, and the feeling that some lectures are ‘rushed’ due to EVS use. 


The enumerated disadvantages, for instance, software and hardware malfunction, 
occur sporadically. Moreover, technical expertise and confidence with using technology usually 
increases with time (King & Robinson, 2009a). An aggregation of the student comments also 
indicated that the time allocated for voting was usually sufficient, and that EVS use for voting 
did not have a negative impact on lecture time, except towards the end of the semester. Further, 
the cases of reported flippancy towards EVS use and swapping of answers seemed to have been 
limited occurrences. 
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In summary, the most important disadvantage mentioned or the one that could have 
potentially high influence on student engagement with learning is the use of ‘sloppy’ questions 
and/or distracters. In answering one of the EVS questions posed during the interview, which 
had also been used in class, K2 pointed out that the distracters (i.e., answer options) were too 
obvious, and that other options could have been used to make the question more challenging: 


K2: Well, this is where the disadvantage of the voting system comes in ‘cause there’ only one 
answer up there with 2 —y. Whereas I’m thinking, for the same example, the same question, was 
up there with the 2—y on the first integral, not on the second integral, I think you might get a few 
more people going ‘ah’. 

I: Which one? 

K2: And making them think ‘do I switch it?’ But obviously for me looking at that I can say that, 
but thats the disadvantage to this method. Whereas if that was a little bit closer for this example, 
number 4, um, the first integral was 0 to 2 —y, I think you’d probably get better results between 
3 and 4 for that reason. 


Discussion 


The results that were earlier presented were based on research conducted in 2009. So this 
is a pertinent question: Why was the research nor published before now, and is technology now a 
significant lever in influencing students’ emotional and behavioural engagement with learning? 
The reason the research presented here is more relevant now (i.e., in 2016) is because there 
has been a significant shift and perceived value on the role that emotions play in influencing 
student learning outcomes at both the school and university levels (Wong, 2015; Salcito, 2012). 
Moreover, technology, especially web-based systems and social media, are now recognized as 
primary influencers of young people and students emotions and behaviours. This is why some 
of the research results presented are still relevant for understanding and designing learning 
tasks that will facilitate positive student engagement with learning. 

For example, the use of laptops and mobile phones during classroom instruction for 
playful purposes that are separate from the learning goals for that class can be very distracting. 
Therefore, attention still plays a pivotal role in (mathematics) classroom discourse. Various 
researchers have examined, for example, the role of attention in helping students learn algebra 
via specialised computer software (Hewitt, 2009) and the interplay between attention, instructor 
pedagogical practices and student engagement with the learning process thus engendered (e.g., 
Mason, 1989; Ainley & Luntley, 2007; Wilson, 2009). Further, research evidence suggests that 
it is hard to maintain attention, within the context of a lecture or presentation, beyond 15 to 20 
minutes at a time (Middendorf & Kalish, 1996; BBC, 2010); although Wilson and Korn (2007) 
disputed the supporting evidence, but not the reality of the attention span itself. Moreover, 
the current generation of students also known as digital natives, or the Google generation are 
notorious multi-taskers (e.g., JISC, 2007; Oblinger, 2008) who find it difficult to concentrate on 
any one thing or task for a considerable length of time. 

Similarly, the favourable perception of the EVS-enabled instructional environment as 
being altruistic is significant because research evidence suggests that students are more likely 
to engage with learning (and also refrain from cheating) when they ‘felt their teacher was more 
concerned with investing in student learning than [in] certification via testing’ (Palazzo, Lee, 
Warnakulasooriya, & Pritchard, 2010). In summary, a favourable perception of the learning 
environment as being altruistic appears to have influenced students to emotionally identify and 
align themselves with the goals and responsibilities expected of them in learning situations 
within that environment. 

Likewise, anonymity is still a valued pedagogical affordance because research evidence 
suggests that, compared to conventional classes, a greater number of students actively 
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participate in EVS-enabled classes, as measured by the number of students who respond to the 
(EVS) questions posed in class (e.g. Fies & Marshall, 2006; Bruff, 2009). One reason for this is 
that students could answer questions without their peers knowing how they answered or what 
choices they selected. This anonymity feature thus helps increase the confidence of students in 
volunteering answers to questions posed by an instructor. 

Meanwhile, it is important to highlight the possible reasons for the favourable student 
perception of the positive learning affordance of the use of EVS. The favourable perception 
of the EVS-enabled instructional environment as being altruistic is perhaps attributable to two 
reasons. First, students in the United Kingdom typically do not have to pay to use the handsets. 
So the handsets are more likely to be seen from an altruistic perspective. In contrast, students 
at many institutions in the USA typically have to pay to use the EVS systems (e.g., King& 
Robinson, 2009a), and this often has a negative correlation with the perceived educational 
value of using clickers for instruction (e.g., Bugeja, 2008). 

Moreover, classroom observation of EVS use in lectures lends credence to the notion that 
student interest in EVS use appears to have moved beyond the initial or transient, ‘situational’ 
interest phase, based on the novel attraction of the EVS contraption, to a more stable orientation 
or ‘personal’ interest phase (Krapp, Hidi & Renninger, 1992). This is being inferred because 
students apparently still displayed positive attitudes towards EVS use, despite having been 
exposed to the technology for two academic years. More recent research (e.g., OECD, 2015b; 
Stover et al., 2015; Voelkel & Bennett, 2014) also buttresses this claim. 

One question that may be asked is how the different elements of engagement interact. A 
valid response to that question would be that emotional, behavioural and cognitive engagement 
are interrelated. The presence of one influences the other two and vice versa (Fredricks, 
Blumenfeld, and Paris, 2004, p. 61). Moreover, students who are not engaged will not learn 
(Watson, De Geest, & Prestage): 


There is a connection between engagement and learning; students cannot learn unless they 
are engaged, and engagement is a combination of social, emotional, intellectual and task 
characteristics. Teachers had to work on all these facets to ensure engagement. All teachers 
believed that learners’ concentration and participation could be developed (p. 12). 


However, it should be noted that the interplay between engagement and learning is a 
complex one and incorporates or necessitates a detailed consideration of themes such as beliefs 
(e.g., teacher’s, student’s and self-efficacy beliefs), motivation and metacognition. 

In addition, research in the field of engagement and mathematical thinking is considering 
how ‘personal meaning, teachers’ emotional knowledge (emotional skills) [and] humour’ 
(Hannula, Panziara & Waege, 2009; see also OECD, 2015b) as engagement constructs may 
influence mathematics teaching and learning. However, a detailed investigation of these issues 
is beyond the scope of this article, although aspects of motivation, using goal theory, will be 
employed in characterising student approaches to learning mathematics in a future study. 


Conclusions 


This research study was designed to provide insights on the research question: How has 
the use of EVS impacted (or otherwise) student engagement with respect to the learning of 
mathematics at university? This research question has been answered through the presentation 
of evidence from interview data about the impact that EVS use has had on behavioural 
(1.e., student attention; student comparison of academic performance; interactivity; learning 
challenge) and emotional engagement (anonymity and altruistic instruction). 

The results show that the EVS utility for students to be able to answer questions 
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anonymously has an enormous impact on the numbers of students responding to questions 
in class. Anonymity helps remove or reduce the fear of being labelled stupid or being 
embarrassed in front of peers, in the event of getting a question wrong. 

The findings also show that EVS use appears to help in stimulating student attention 
because of the realisation that they will subsequently be asked questions after the presentation 
of a topic in class makes them alert and to pay attention during instruction. Similarly, the results 
of the question and answer sessions can help to re-focus student attention by highlighting areas 
of learning deficiencies. Further, instructors can strategically use questions at definite intervals 
during a lecture to maintain student attention. 

Further, EVS use not only facilitates increased student participation in mathematics 
lectures, it also enhances the mode and frequency of interaction that students have with their 
instructor, one another, and the EVS tool. Hence interactivity is significantly increased. This 
enhanced interactivity in turn catalyses a classroom atmosphere that is characterised by a 
positive student competitiveness to get answers right and compare their answers with peers 
during classroom voting on mathematics problems. Similarly, the display of answers to EVS 
questions in a format that is viewed by the whole class enables individual student diagnosis of 
the level of their understanding and performance, relative to the whole class. 

Last, students appear to have a favourable perception of the EVS-enabled instructional 
environment as being altruistic. This favourable perception in turn appears to have influenced 
them to emotionally identify and align themselves with instructional goals. 

But students’ comments show that EVS use is not entirely beneficial, as EVS technology 
malfunction, instances of low student voting, and guessing on questions, can obviate or minimize 
the learning engagement benefits of using EVS. Specifically, the use of mathematics 
questions with low cognitive demand or easy-to-guess answers is seen as 
being detrimental to enhancing student engagement. 

Finally, this research has highlighted that although the behavioural, emotional (and 
cognitive) engagement constructs have been separately examined, in reality, they are 
interconnected. Hence, future research will investigate whether the impact of EVS use on 
engagement, as reported in this study, is stable over time. It would be valuable to conduct a 
longitudinal study on how (if at all), and the degree to which, the pedagogical alignment of EVS 
with student learning outcomes catalyses learning beyond the classroom, e.g., in informal and 
virtual learning contexts. 


Note 


The research described in this article was completed while the author was at Loughborough 
University, United Kingdom 
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